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DETAILED ACTION 

Drawings 

The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: Fig. 1 (element 10, disclosed as element 104 in Specification), Fig. 3 
(element 300), Fig. 4 (element 400). Corrected drawing sheets in compliance with 37 
CFR 1.121(d), or amendment to the specification to add the reference character(s) in 
the description in compliance with 37 CFR 1.121(b) are required in reply to the Office 
action to avoid abandonment of the application. Any amended replacement drawing 
sheet should include all of the figures appearing on the immediate prior version of the 
sheet, even if only one figure is being amended. Each drawing sheet submitted after the 
filing date of an application must be labeled in the top margin as either "Replacement 
Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the examiner does not accept 
the changes, the applicant will be notified and informed of any required corrective action 
in the next Office action. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-4, 7-8, 10 and 13-18 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Matusik et al., "Polyhedral Visual Hulls for Real-Time Rendering", 
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Proceedings of Twelfth Eurographics Workshop on Rendering, 2001, pages 115-125, in 
view of Deering U.S. Patent No. 5867167. 

Referring to claim 1 , Matusik et al. teaches a method for providing a virtual reality 
environment, comprising: acquiring concurrently, with a plurality of cameras, a plurality 
of sequences of input images of a 3D object, each camera having a different pose 
(page 115, Section 1 Introduction, 4 th paragraph; page 123, Section 4 Real-Time 
System, 1 st paragraph); reducing the plurality of sequences of images to a differential 
stream (page 123, Section 4 Real-Time System, 1 st paragraph); rendering the 3D model 
as a sequence of output image of the 3D object from an arbitrary point of view while 
acquiring and reducing the plurality of sequences of images and maintaining the 3D 
model in real-time (page 124, Section 4 Real-Time System, 1 st paragraph) but does not 
specifically teach reducing the plurality of sequences of images to a differential stream 
of 3D operators and operands and maintaining a 3D model of point samples 
representing the 3D object from the differential stream, in which each point sample of 
the 3D model has 3D coordinates and intensity information. 

Deering teaches reducing the plurality of sequences of images to a differential 
stream of 3D operators and operands (Figs. 4 (A-K), 9 (elements 410-450) and 14 (A-J); 
column 12, lines 35-41; column 17, lines 35-42, i.e. normal, position, and color 
information for the plurality of images are compressed into a differential stream of 3D 
operators and operands) and maintaining a 3D model of point samples representing the 
3D object from the differential stream, in which each point sample of the 3D model has 
3D coordinates and intensity information (Fig. 14B; column 8, lines 18-20; column 9, 
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lines 50-53, i.e. position information contains the 3D x, y, and z coordinates and normal 
information indicates intensity information). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
reducing the plurality of sequences of images to a differential stream of 3D operators 
and operands and maintaining a 3D model of point samples representing the 3D object 
from the differential stream, in which each point sample of the 3D model has 3D 
coordinates and intensity information thereby providing a method for compressing data 
in real-time that substantially reduces the bit-size of the file to be transmitted with little 
loss in displayed object quality at low cost (Deering column 5, lines 18-24 and 55-65). 

Referring to claim 2, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, in which the acquiring and 
reducing are performed at a first node, and the rendering and maintaining are 
performed at a second node, and further comprising: transmitting the differential stream 
from the first node to the second node by a network (page 123, Section 4 Real-Time 
System, 1 st paragraph). 

Referring to claim 3, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, wherein the object is moving with 
respect to the plurality of cameras (page 115, Section 1 Introduction, 2 nd and 4 th 
paragraphs; page 116, Section 1.1 Previous Work, 6 th paragraph; page 123, Section 4 
Real-Time System, 1 st paragraph, i.e. it is understood that the use of four calibrated 
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video cameras capturing a video stream and performing processing steps on the video 
stream indicates movement of the foreground object). 

Referring to claim 4, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, wherein the reducing further 
comprises: segmenting the object from a background portion in a scene; and discarding 
the background portion (page 123, Section 4 Real-Time System, 1 st paragraph). 

Referring to claim 7, the rationale for claims 1 and 6 is incorporated herein, 
Matusik et al., as modified by Deering above, teaches the method of claim 1, comprising 
reducing the sequences of images to a differential stream having operators but does not 
specifically teach wherein the operators include insert, delete, and update operators. 

Deering teaches wherein the operators include update and insert operators 
(column 13, lines 19-54; columns 20-21, lines 50-14; column 21, lines 17-29; column 
22, lines 6-14 and 42-25; column 23, lines 29-60, i.e. normal and color values are 
updated by replacing the previous values stored in the mesh buffer with new values 
labeled as current values, Deering further teaches that other instruction sets may be 
used and that new vertices may be inserted into the mesh buffer). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the operators include update and insert operators thereby providing a method 
for compressing data in real-time that substantially reduces the bit-size of the file to be 
transmitted with little loss in displayed object quality at low cost (Deering column 5, lines 
18-24 and 55-65) and thus improving effective bandwidth for a graphics accelerator 
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system, including shared virtual reality display environments (Deering column 17, lines 
7-13). 

Further, although Deering does not specifically teach wherein the operators 
include delete operators said operators are well known in the art, therefore it would 
have been obvious to one skilled in the art at the time of the invention to include said 
operators in the method of Matusik et al., because it would provide a more accurate 
rendering of the image. 

Referring to claim 8, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, but does not specifically teach 
wherein the associated operand includes a 3D position and color as attributes of the 
corresponding point sample. 

Deering teaches wherein the associated operand includes a 3D position and 
color as attributes of the corresponding point sample (Figs. 4K, 10 (element 630) and 14 
(A-B)). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the associated operand includes a 3D position and color as attributes of the 
corresponding point sample thereby providing a method for compressing data in real- 
time that substantially reduces the bit-size of the file to be transmitted with little loss in 
displayed object quality at low cost (Deering column 5, lines 18-24 and 55-65) and thus 
improving effective bandwidth for a graphics accelerator system, including shared virtual 
reality display environments (Deering column 17, lines 7-13). 
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Referring to claim 10, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, in which the point samples are 
maintained on a per camera basis (page 121, Section 3 View-Dependent Texturing, 2 nd - 
4 th paragraphs, i.e. sparse point sampling each image by sampling along the 3D rays 
and forming a texture matrix associated with camera / is understood to be maintaining 
the point samples on a per camera basis). 

Referring to claim 13, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1 , in which the point samples are 
rendered as polygons (pages 121-122, Section 3 View-Dependent texturing, 3 rd 
paragraph). 

Referring to claim 14, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, further comprising sending a 
silhouette image corresponding to a contour of the 3D object in the differential stream 
for each reduced image (page 123, Section 4 Real-Time System, 1 st paragraph). 

Referring to claim 15, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, in which the differential stream is 
compressed (page 123, Section 4 Real-Time System, 1 st paragraph). 

Referring to claim 16, the rationale for claim 1 is incorporated herein, Matusik et 
al M as modified above, teaches the method of claim 1, but does not specifically teach 
wherein the associated operand includes a normal of the corresponding point sample. 

Deering teaches wherein the associated operand includes a normal of the 
corresponding point sample (Figs. 4K, 10 (element 630) and 14 (A-B)). 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the associated operand includes a normal of the corresponding point sample 
thereby providing a method for compressing data in real-time that substantially reduces 
the bit-size of the file to be transmitted with little loss in displayed object quality at low 
cost (Deering column 5, lines 18-24 and 55-65) and thus improving effective bandwidth 
for a graphics accelerator system, including shared virtual reality display environments 
(Deering column 17, lines 7-13). 

Referring to claim 17, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, in which the associated operand 
includes reflectance properties of the corresponding point sample. 

Deering teaches wherein the associated operand includes reflectance properties 
of the corresponding point sample (column 8, lines 49-54). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the associated operand includes reflectance properties of the corresponding 
point sample thereby providing a method for compressing data in real-time that 
substantially reduces the bit-size of the file to be transmitted with little loss in displayed 
object quality at low cost (Deering column 5, lines 18-24 and 55-65) and thus improving 
effective bandwidth for a graphics accelerator system, including shared virtual reality 
display environments (Deering column 17, lines 7-13). 
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Referring to claim 18, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, in which pixels of each image are 
classified as either foreground or background pixels, and in which only foreground 
pixels are reduced to the differential stream (page 123, Section 4 Real-Time System, 1 st 
paragraph). 

Claims 9, 12, and 19-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Matusik et al., "Polyhedral Visual Hulls for Real-Time Rendering", 
Proceedings of Twelfth Eurographics Workshop on Rendering, 2001, pages 115-125, in 
view of Deering U.S. Patent No. 5867167 as applied to claim 1 above, and further in 
view of Pauly et al., "Spectral Processing of Point-Sampled Geometry", Proc. of 28th 
Annual Conf. on Computer Graphics and Interactive Techniques, SIGGRAPH 2001, 
ACM Press, New York, NY, 379-386. 

Referring to claim 9, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, but does not specifically teach 
wherein the point samples are rendered with point splatting. 

Pauly et al. teaches wherein the point samples are rendered with point splatting 
(page 382, Section 3 Scattered Data Approximation, Regular Sampling, 2 nd paragraph; 
page 384, Section 7 reconstruction, Blending function). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the point samples are rendered with point splatting thereby allowing for direct 
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processing or manipulation of point-sampled geometry without a need for polygonal 
meshes (page 379, Section 1 Introduction, 1 st paragraph). 

Referring to claim 12, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, but does not specifically teach 
estimating a local density for each point sample. 

Pauly et al. teaches estimating a local density for each point sample (page 379, 
Section 1 Introduction, 3 rd paragraph). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
estimating a local density for each point sample thereby reducing the complexity of 
overly dense sample models (page 379, Section 1 Introduction, 3 rd paragraph). 

Referring to claim 19, the rationale for claims 1 and 8 is incorporated herein, 
Matusik et al., as modified above, teaches the method of claims 1 and 8 wherein 
attributes are assigned to each point sample, but does not specifically teach wherein the 
attributes are altered while rendering. 

Pauly et al. teaches wherein the attributes are altered while rendering (page 384, 
Section 7 Reconstruction, Blending Normals). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the attributes are altered while rendering thereby allowing for direct processing 
or manipulation of point-sampled geometry without a need for polygonal meshes (page 
379, Section 1 Introduction, 1 st paragraph). 
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Referring to claim 20, the rationale for claim 19 is incorporated herein, Matusik et 
al. t as modified above, teaches the method of claim 19, but does not specifically teach 
wherein the point attributes are organized in a vertex array that is transferred to a 
graphics memory during the rendering. 

Deering teaches wherein the point attributes are organized in a vertex array that 
is transferred to a graphics memory during the rendering (column 6, lines 35-41; column 
7, lines 39-55, i.e. it is understood that the compressed geometry containing the point 
color and normal attributes is stored in a vertex array in the mesh buffer). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the point attributes are organized in a vertex array that is transferred to a 
graphics memory during the rendering thereby providing a method for compressing data 
in real-time that substantially reduces the bit-size of the file to be transmitted with little 
loss in displayed object quality at low cost (Deering column 5, lines 18-24 and 55-65) 
and thus improving effective bandwidth for a graphics accelerator system, including 
shared virtual reality display environments (Deering column 17, lines 7-13). 

Claims 5 and 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Matusik et al., "Polyhedral Visual Hulls for Real-Time Rendering", Proceedings of 
Twelfth Eurographics Workshop on Rendering, 2001, pages 1 15-125, in view of Deering 
U.S. Patent No. 5867167 as applied to claim 1 above, and further in view of Kanade et 
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al. "Virtualized Reality: Constructing Virtual Worlds from Real Scenes", IEEE Computer 
Society, pp.34-47, January - March 1997. 

Referring to claim 5, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1 , but does not teach wherein the 
reducing further comprises selecting, at any one time, a set of active cameras from the 
plurality of cameras. 

Kanade et al. teaches wherein the reducing further comprises selecting, at any 
one time, a set of active cameras from the plurality of cameras (page 37, Section 
Camera Clusters, i.e. it is understood that the reference camera and 3-6 immediate 
neighbor cameras are the active cameras). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include a 
plurality of cameras wherein the reducing further comprises selecting, at any one time, a 
set of active cameras from the plurality of cameras because using many cameras 
improves the extent and accuracy of stereo while selecting a set of active cameras from 
the plurality of cameras reduces the computational costs. 

Referring to claim 1 1, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, but does not specifically teach 
wherein the rendering combines the sequence of output images with a virtual scene. 

Kanade et al. teaches wherein the rendering combines the sequence of output 
images with a virtual scene (page 45, Combining virtual and virtualized environments). 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the rendering combines the sequence of output images with a virtual scene 
thereby allowing virtual foreground objects to be introduced into the virtualized 
environment. 

Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Matusik 
et al., "Polyhedral Visual Hulls for Real-Time Rendering", Proceedings of Twelfth 
Eurographics Workshop on Rendering, 2001, pages 115-125, in view of Deering U.S. 
Patent No. 5867167 as applied to claim 1 above, and further in view of Lee et al. U.S. 
Patent No. 5684887. 

Referring to claim 6, the rationale for claim 1 is incorporated herein, Matusik et 
al., as modified above, teaches the method of claim 1, but does not specifically teach 
wherein the differential stream of 3D operators and associated operands reflect 
changes in the plurality of sequences of images. 

Lee et al. teaches wherein the differential stream of 3D operators and associated 
operands reflect changes in the plurality of sequences of images (column 9, lines 40-67; 
column 10, lines 26-36). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method of Matusik et al. to include 
wherein the differential stream of 3D operators and associated operands reflect 
changes in the plurality of sequences of images thereby providing a method wherein a 
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slowly changing background can be updated every frame and the object boundary can 
be recovered instantaneously by a difference operation (Lee et al. column 10, lines 1-3). 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

The following patents are cited to further show the state of the art with respect to 
operands and operators/opcodes. 

Deering U.S. Patent No. 5793371 

Moorer U.S. Patent No. 5867167 

Deering U.S. Patent No. 5842004 

Karol et al. U.S. Patent No. 6122275 

Deering U.S. Patent No. 6459429 

The following patents are cited to further show the state of the art with respect to 
difference frames/images, client-server nodes, data compression, and data 
transmission. 

Cohen-Or U.S. Patent No. 6307567 

Mannetal. U.S. Patent No. 6330281 

Kelly U.S. Patent No. 2003/0219163 

The following patents are cited to further show the state of the art with respect to 
background subtraction and composition of object and scene images. 
Lippman et al. U.S. Patent No. 5262856 
Miller et al. U.S. Patent No. 5999641 
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Deering U.S. Patent No. 5867167 
Kanade et al. U.S. Patent No. 6084979 
Matsumoto et al. U.S. Patent No. 6356272 
Bordeset al. U.S. Patent No. 6909747 

The following non-patent literature cited to further show the state of the art. 

Gross et al., "Blue-c: a Spatially Immersive Display and 3D Video Portal for 
Telepresence", ACM Trans. Graph. 22, 3 (Jul. 2003), pages 819-827. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Roberta Prendergast whose telephone number is (571) 
272-7647. The examiner can normally be reached on M-F 7:00-4:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on (571) 272-7782. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



RP 



^kQA^U — 

ULKA J. CHAUHAN 
PRIMARY EXAMINER 



